1. Field of the Invention
The present invention relates to improvements in neural networks, hardware for carrying out the functions of a neural network, neural network processors, and neural network pattern recognition apparatuses.
2. Description of the Related Art
The neural network, namely, a system of recognizing predetermined input information and providing the results of the recognition which is, based on a conception entirely differing from those on which conventional methods are based, has been developed and applied to various fields. The neural network is a model of a human brain, which can be realized in various ways.
The neural network has been proposed in a mathematical algorithm having a complex structure. Accordingly, the conventional neural network has been realized by computer simulation. Computer simulation, however, operates at a comparatively low processing speed, which is a problem in some practical applications. Recently, comprehensive study of the neural network has been made and pieces of hardware for realizing the neural network have been proposed. However, the proposed hardware deals with neural networks having only one or two layers.
A Neocognitron is one model of a neural network. Only a few studies have been made on the development of hardware for realizing a Neocognitron, because a Neocognitron is a neural network with a complex structure. Hardware for realizing a Neocognitron has been reported in a paper published by MIT. This paper was published in the poster session of NIPS (Neural Information Processing & Systems) '90. The hardware is simple in structure comprising, in combination, 143 CCD arrays and seven MDACs (multiplier DA converters). Most circuits employed in the hardware are digital circuits. Basically, both input data and coefficient data are stored in the digital circuits, and the semianalog MDACs carry out multiplication. Since the method of making this system was not able to fabricate division circuits satisfactorily, only a first layer was realized. The hardware has a small degree of integration of seven multipliers in 29 mm.sup.2.
Thus, the realization of a neural network in hardware has encountered many difficulties and hence methods have been studied for the high-speed simulation of a neural network having three or more layers. One of the methods simulates the neural network using a program to be executed by parallel processing computers. However, if this method is employed, it often occurs that the computational topology of the neural network does not coincide with the architecture of each computer, and the efficiency of data transmission between the processing elements is reduced. Even if parallel computers having many processing elements are employed for high-speed simulation, it is difficult to improve the cost performance.